CMAQ Data Available on the Cloud through Amazon’s Open Data Program
Citation:
Sidi, F., K. Foley, G. Pouliot, A. Eyth, E. Adams, AND S. Arunachalam. CMAQ Data Available on the Cloud through Amazon’s Open Data Program. CMAS Conference 2023, Chapel Hill, NC, October 16 - 18, 2023.
Impact/Purpose:
This poster is aimed at advertising the current suite of CMAQ data products released via Amazon Web Services (AWS). This will include instructions on how to download as well as possibly use the data on AWS. The switch to using AWS as a data sharing hub will hopefully resolve some data management and transfer issues that earth system modeling groups like CMAQ have been facing.
Description:
With the emergence of cloud computing technologies, many earth system modeling groups have explored cloud-based solutions for managing large modeling input and output datasets. At EPA large (100s of GB to 10s of TB) modeling datasets from the Community Multiscale Air Quality (CMAQ) modeling system were previously shared via shipped hard drives to individual groups. This method can be slow and expensive and does not provide access to the data to the larger modeling community. Large modeling data have also been shared online via the CMAS Data Warehouse Google Drive which allows for free downloads at relatively fast data transfer speeds. However, CMAQ community members have sometimes had data access issues when trying to download many large datasets from the Google Drive. In response to this, the CMAS Center partnered with Amazon Web Services (AWS) to host air quality modeling data on the cloud through Amazon’s Open Data Program. The major benefits of this program include free hosting of popular large datasets (e.g., emissions modeling platforms, annual sets of CMAQ inputs for multiple years and chemical mechanisms, post-processed CMAQ outputs), waived egress fees for users downloading data, and the ability to do numerical modeling using cloud computing services thus avoiding the need to download the large datasets to a local system. We will present a summary of the air quality datasets available on the CMAS AWS Data Warehouse and how to download the data or use AWS to run a CMAQ simulation on the cloud.